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Abstract 

We initiate the study of metric embeddings with outliers. Given some metric space (A, p) 
we wish to find a small set of outlier points K C X and either an isometric or a low-distortion 
embedding of (A \ K , p) into some target metric space. This is a natural problem that captures 
scenarios where a small fraction of points in the input corresponds to noise. 

For the case of isometric embeddings we derive polynomial-time approximation algorithms for 
minimizing the number of outliers when the target space is an ultrametric, a tree metric, or some 
constant-dimensional Euclidean space. The approximation factors are 3, 4 and 2, respectively. 
For the case of embedding into an ultrametric or tree metric, we further improve the running 
time to 0(n 2 ) for an n-point input metric space, which is optimal. We complement these upper 
bounds by showing that outlier embedding into ultrametrics, trees, and d-dimensional Euclidean 
space for any d> 2 are all NP-hard, as well as NP-hard to approximate within a factor better 
than 2 assuming the Unique Game Conjecture. 

For the case of non-isometries we consider embeddings with small distortion. We present 
polynomial-time bi-criteria approximation algorithms. Specifically, given some e > 0, let k £ 
denote the minimum number of outliers required to obtain an embedding with distortion e. For 
the case of embedding into ultrametrics we obtain a polynomial-time algorithm which computes 
a set of at most 3 k e outliers and an embedding of the remaining points into an ultrametric with 
distortion 0(£logn). Finally, for embedding a metric of unit diameter into constant-dimensional 
Euclidean space we present a polynomial-time algorithm which computes a set of at most 2 k £ 
outliers and an embedding of the remaining points with distortion 0{ y/e). 
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1 Introduction 


Metric embeddings provide a framework for addressing in a unified manner a variety of data-analytic 
tasks. Let X = (X,p), y = ( Y, p') be metric spaces. At the high level, a metric embedding is a 
mapping / : X —» Y that either is isometric or preserves the pairwise distances up to some small 
error called the distortior^ The corresponding computational problem is to decide whether an 
isometry / exists or, more generally, to find a mapping / with minimum distortion. The space y 
might either be given or it might be constrained to be a member of a collection of spaces, such 
as trees, ultrametrics, and so on. The problems that can be geometrically abstracted using this 
language include phylogenetic reconstruction (e.g. via embeddings into trees eeihj or ultrametrics 
Emm), visualization (e.g. via embeddings into constant-dimensional Euclidean space [HI EH nm E21 
nsEuniEZ!), and many more (for a more detailed exposition we refer the reader to [261 [25]). 

Despite extensive research on the above metric embedding paradigm, essentially nothing is 
known when the input space X can contain outliers. This scenario is of interest for example in 
applications where outliers can arise from measurement errors. Another example is when real-world 
data does not perfectly fit a model due to mathematical simplifications of physical processes. 

We propose a generalization of the above high-level metric embedding problem which seeks to 
address such scenarios: Given X and y we wish to find some small K C X and either an isometric 
or low-distortion mapping / : X \ I\ —> Y. We refer to the points in K as outliers. 

We remark that it is easy to construct examples of spaces X, y where any embedding / : X —» Y 
has arbitrarily large distortion (for any “reasonable” notion of distortion), yet there exists x E X 
and an isometry f : X \ {x} —> Y. Thus new ideas are needed to tackle the more general metric 
embedding problem in the presence of outliers. 

1.1 Our contribution 

Approximation algorithms. We focus on embeddings into ultrametrics, trees, and constant¬ 
dimensional Euclidean space. We first consider the problem of computing a minimum size set of 
outliers such that the remaining point-set admits an isometry into some target space. We refer to 
this task as the minimum outlier embedding problem. 

Outlier embeddings into ultrametrics. It is well-known that a metric space is an ultrametric 
if and only if any 3-point subset is an ultrametric. We may therefore obtain a 3-approximation 
as follows: For all ( x,y,z ) G A 3 , if the triple ( x,y,z ) is not an ultrametric then remove x, y, 
and z from X. It is rather easy to see that this gives a 3-approximation for the minimum outlier 
embedding problem into ultrametrics (as for every triple of points that we remove, at least one of 
them must be an outlier in any optimal solution), with running time 0(n 3 ). By exploiting further 
structural properties of ultrametrics, we obtain a 3-approximation with running time 0(n 2 ). We 
remark that this running time is optimal since the input has size 0(n 2 ) and it is straightforward 
to show that any 3-approximation has to read all the input (e.g. even to determine whether X is 
an ultrametric, which corresponds to the case where the minimum number of outliers is zero). 

Outlier embeddings into trees. Similarly to the case of ultrametrics, it is known that a space 
is a tree metric if and only if any 4-point subset is a tree metric. This similarly leads to a 4- 
approximation algorithm in 0(n 4 ) time. We further improve the running time to 0(n 2 ), which is 

Warious definitions of distortion have been extensively considered, including multiplicative, additive, average, and 
£ p distortion, as well as expected distortion when the map / is random [6j. 
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also optimal. However, obtaining this improvement is significantly more complicated than the case 
of ultrametrics. 

Outlier embeddings into . It is known that for any d > 1 any metric space admits an isometric 
embedding into d-dimensional Euclidean space if and only if any subset of size d + 3 does [33]. This 
immediately implies a (d + 3)-approximation algorithm for outlier embedding into d-dimensional 
Euclidean space with running time 0(n d+ 3 ), for any d > 1. Using additional rigidity properties of 
Euclidean space we obain a 2-approximation with the same running time. 

Hardness of approximation. We show that, assuming the Unique Games Conjecture [28], the 
problems of computing a minimum outlier embedding into ultrametrics, trees, and d-dimensional 
Euclidean space for any d > 2, are all NP-hard to approximate within a factor of 2 — u, for any 
v > 0. These inapproximability results are obtained by combining reductions from Vertex Cover to 
minimum outlier embedding and the known hardness result for the former problem [29]. Note that 
for the case of embedding into d-dimensional Euclidean space for any d > 2 this inapproximability 
result matches our upper bound. 

Bi-criteria approximation algorithms. We also consider non-isometric embeddings. All our 
results concern distortion. For some outlier set K C X , the distortion of some map / : 
X \ K —> Y is defined to be 

sup \p(x,y) - p'(f(x),f{y))\ . 

x,y&X\K 

In this context there are two different objectives that we wish to minimize: the number of outliers 
and the distortion. For a compact metric space Z = ( Z , pz ), denote by A (Z) = sup., z / eZ Pz(z , z') 
the diameter of Z. 

Definition 1.1 ((e, fc)-Outlier embedding). We say that X admits a ( s , /c)-outlier embedding into 
y if there exists K C X with \K\ < k and some f : X\K —>• Y with distortion at most eA(X). 
We refer to K as the outlier set that witnesses a (e, k)-outlier embedding of X. 

Note that the multiplication of the distortion by A (A) is to make the parameter e scale-free. 
Since an isometry can be trivially achieved by removing all but one points; thus the above notion 
is well-defined for all e > 0. We now state our main results concerning bi-criteria approximation: 

Bi-criteria outlier embeddings into ultrametrics: We obtain a polynomial-time algorithm which 
given an n-point metric space X and some e > 0 such that X admits a (s, fc)-outlier embedding 
into an ultrametric, outputs a 0(elogn, 3fc)-outlier embedding into an ultrametric. 

Bi-criteria outlier embeddings into We present an algorithm which given an n-point metric 
space X and some e > 0 such that X admits a (e, fe)-outlier embedding in M d , outputs a (0(t/e), 2k)- 
outlier embedding of X into M rf . The algorithm runs in time 0(n rf+3 ). 

Bi-criteria outlier embeddings into trees: Finally we mention that one can easily derive a bi¬ 
criteria approximation for outlier embedding into trees by the work of Gromov on h-hyperbolicity 
m\ (see also P3D- Formally, there exists a polynomial-time algorithm which given a metric space X 
and some e > 0 such that X admits a (e, fc)-outlier embedding into a tree, outputs a (0(e logn), 4k)- 
outlier embedding into a tree. Let us briefly outline the proof of this result: h-hyperbolicity is a 
four-point condition such that any 5-hyperbolic space admits an embedding into a tree with i^ 
distortion 0(6 log n), and such an embedding can be computed in polynomial time. Any metric 
that admits an embedding into a tree with distortion e is 0(e)-hyperbolic. Thus by removing all 
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4-tuples of points that violate the 0(e)-hyperbolicity condition and applying the embedding from 
[23] we immediately obtain an (O(elogn), 4/c)-outlier embedding into a tree. We omit the details. 

1.2 Previous work 

Over the recent years there has been a lot work on approximation algorithms for minimum distortion 
embeddings into several host spaces and under various notions of distortion. Perhaps the most 
well-studied case is that of multiplicative distortion. For this case, approximation algorithms and 
inapproximability results have been obtained for embedding into the line M CHI El [22] , constant¬ 
dimensional Euclidean space [Hum na m uni , trees m IT5| . ultrametrics [3], and other graph- 
induced metrics m- We also mention that similar questions have been considered for the case 
of bijective embeddings [35, [Ml [27, 13 l30j . Analogous questions have also been investigated for 
average [18], additive [5] , i v [2] , and distortion J2DJ [1]. 

Similar in spirit with the outlier embeddings introduced in this work is the notion of embeddings 
with slack [13 da EE]. In this scenario we are given a parameter e > 0 and we wish to find an 
embedding that preserves (1 — e)-fraction of all pairwise distances up to a certain distortion. We 
remark however that these mappings cannot in general be used to obtain outlier embeddings. This 
is because typically in an embedding with slack the pairwise distances that are distorted arbitrarily 
involve a large fraction of all points. 

1.3 Discussion 

Our work naturally leads to several directions for further research. Let us briefly discuss the most 
prominent ones. 

An obvious direction is closing the gap between the approximation factors and the inapprox¬ 
imability results for embedding into ultrametrics and trees. Similarly, it is important to understand 
whether the running time of the 2-approximation for embedding into Euclidean space can be im¬ 
proved. More generally, an important direction is understanding the approximability of outlier 
embeddings into other host spaces, such as planar graphs and other graph-induced metrics. 

In the context of bi-criteria outlier embeddings, another direction is to investigate different 
notions of distortion. The case of distortion studied here is a natural starting point since it is 
very sensitive to outliers. It seems promising to try to adapt existing approximation algorithms for 
i p , multiplicative, and average distortion to the outlier case. 

Finally, it is important to understand whether improved guarantees or matching hardness results 
for bi-criteria approximations are possible. 

2 Definitions 

A metric space is a pair X = ( X , p) where X is a set and p : X x X —> M>o such that (i) for any 
!,!/£ X, p(x , y) = p(y, x ) > 0, (ii) p(x , y) = 0 if and only if x = y, and (iii) for any x,y,z e X, 
p(x,z) < p(x,y) + p(y,z). Given two metric spaces X = (X,px) and y = (Y,py), an embedding 
of X into y is simply a map p : X —>• Y, and p is an isometric embedding if for any x,x' 6 X , 
px{x,x') = p Y (p{x),p(x')). 

In this paper our input is an n-point metric ( X , p), meaning that A is a discrete set of cardinality 
n. Given an ?r-point metric space X = ( X , p) and a value a > 0, we denote by X + a the metric 
space (A, p') where for any x / y £ A we have p'(x , y) = p(x , y) + a. 
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Definition 2.1 (Ultrametric space). A metric space (X,p) is an ultrametric (tree) space if and 
only if the following three-point condition holds for any x,y,z£X: 

p(x, y) < max{p(x, z),p(z, y)}. (1) 

Definition 2.2 (Tree metric). A metric space (X,p) is a tree metric if and only if the following 
four-point condition holds for any x,y,z,w G X: 

p{x, y) + p(z, w ) < max{p(r, z) + p(y, w),p(x, w) + p(y, 2 )}. (2) 

An equivalent formulation of the four-point condition is that for all x, y,z,w G X, the largest two 
quantities of the following three terms are equal: 

p(x,y) + p(z,w), p(x,z) + p(y,w), p{x,w) + p(y,z). (3) 

In particular, an n-point tree metric ( X , p) can be realized by a weighted tree T such that there 
is a map <p : X -» V(T) into the set of nodes V(T) of T, and that for any x,y G X, the shortest 
path distance dx(p i (x),ip(y)) in T equals p(x,y). In other words, <p is an isometric embedding of 
(X,p) into the graphic tree metric (T, dr)- An ultrametric (X, p) is in fact a special case of tree 
metric, where there is an isometric embedding ip : X —> V(T) to a rooted tree (T,dx) such that 
p>(X) are leaves of T and all leaves are at equal distance from the root of T. 

3 Approximation algorithms for outlier embeddings 

In this section we present approximation algorithms for the minimum outlier embedding problem 
for three types of target metric spaces: ultrametrics, tree metrics, and Euclidean metric spaces. 
We show in Appendix |B| that finding optimal solutions for each of these problems is NP-hard (and 
roughly speaking hard to approximate within a factor of 2 as well). In the cases of ultrametric and 
tree metrics, it is easy to approximate the minimum outlier embedding within constant factor in 
0(n 3 ) and 0(n 4 ) time, respectively. The key challenge (especially for embedding into tree metric) 
is to improve the time complexity of the approximation algorithm to 0(n 2 ), which is optimal. 

3.1 Approximating outlier embeddings into ultrametrics 

Theorem 3 . 1 . Given an n-point metric space (X,p), there exists a 3- approximation algorithm for 
minimum outlier embedding into ultrametrics, with running time 0(n 2 ). 

Proof. We can obtain a polynomial-time 3-approximation algorithm as follows: For each triple of 
points (x, y, z) G X 3 , considered in some arbitrary order, check whether it satisfies 0. If not, then 
remove x, y, and z from X and continue with the remaining triples. Let K be the set of removed 
points. For every triple of points removed, at least one must be in any optimal solution; therefore 
the resulting solution K is a 3-approximation. The running time of this method is 0(n 3 ). We next 
show how to improve the running time to 0(n 2 ). 

Let X = {xi,..., x n }. We inductively compute a sequence Xq, ..., X n C X, where Aq is set to 
be Xq = 0. Given X,;_i for some i G {1,..., n}, assuming the invariance that A,;_i is an ultrametric, 
we compute Xj as follows. We check whether (A',_i U {xi}, p) is an ultrametric. If it is, then we set 
Xi = X;_ 1 U {.Tj}. Otherwise, there must exist ( x,y,z ) G (X;_ 1 U {xj}) 3 that violates ([!]). Since 
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(Xi-i, p) is an ultrametric, it follows that every such triple must contain Xj. Therefore it suffices 
to show how to quickly find y,z E Xi-\ such that ( Xi,y,z ) violates if they exist. To this end, 
let x* be a nearest neighbor of Xj in Xj_i, that is x* = argmin a , gX ._ i { / o(xj, x)}, where we brake 
ties arbitrarily. Instead of checking x t against all possible y,z from Xj_i, we claim that 0 holds 
for all ( Xi , y , z) with y,z E Xj_i if and only if for all w E Xj_i we have 

(i) p(xi,x*) < max{p(xi,w),p(x*,w)} and (ii) p(xi, iv) = p(x*, w). (4) 


Indeed, assume that (i) and (ii) above hold for all w E Xj_i, yet there exist some y, z E Xj_i 
such that ( Xi,y,z ) violates ([!]), say w.l.o.g., p(xi,y) > max{p(xi, z), p(y, z)}. Then by (ii) above, 
we have p(xi,y) = p(x*,y ) and p(xi,z) = p(x*,z), implying that p(x*,y) > max{p(x*, z), p(y, z)}. 
Hence ( x*,y,z ) also violates ([!]), contradicting the fact that (Xj_i , p) is an ultrametric. Hence no 
such y,z can exist, and ( fTT)] ) is sufficient to check whether Xj_i U {xj} induces an ultrametric or 
not. 

Finally, we can clearly check in time 0(n ) whether both conditions in (10) hold for all w E Xj_i . 
If either (i) or (ii) in (10) fails then (xi,x*,w) violates 0< which concludes the proof. □ 


3.2 Approximating outlier embeddings into trees 

We now present a 4-approximation algorithm for embedding a given n-point metric space into a tree 
metric with a minimum number of outliers. Using the four-point condition ([2]) in Definition |2.2| it 
is fairly simple to obtain a 4-approximation algorithm for the problem with running time 0(nr) as 
follows: Check all 4-tuples of points x,y,z,w E X. If the 4-tuple violates the four-point condition, 
then remove x,y,z,w from X. It is immediate that for any such 4-tuple, at least one of its points 
much be an outlier in any optimal solution. It follows that the result is a 4-approximation. 

We next show how to implement this approach in time 0(n 2 ). The main technical difficult is 
in finding a set of violating 4-tuples quickly. The high-level description of the algorithm is rather 
simple, and is as follows. Let (X,p) be the input metric space where X = {x \,..., x n }. Set 
X\ = {xi}. For any i = 2,..., n, we inductively define Xi C X. At the beginning of the z-th 
iteration, we maintain the invariance that (Xi-i, p) is a tree metric. If (Xj_i U {xi},p) is a tree 
metric, then we set Xi = Xj_i U {xi}. Otherwise there must exist yi,Zi,Wi E Xj_i such that the 
4-tuple (xi,yi, Zi,Wi) violates the four-point condition; we set Xi = X. L _\ \ {yi, Zi,Wi}. 

To implement this idea in 0(n 2 ) time, it suffices to show that for any i = 2,... ,n, given Xi- 1 , 
we can compute X 1, in time 0(n). The algorithm will inductively compute a collection of edge- 
weighted trees Ti,..., T n . with T\ simply being the graph with V(T\) = {xi}, and maintain the 
following invariants for each i E [1, n\: 

(1-1) Xj. C V{Ti) and all leaves of 7) are in Xj. (Xi, p) embeds isometrically into (7), ci^); that is, 
the shortest-path metric of T* agrees with p on X*: for any x,y E Xj, dT t (x,y) = p(x,y). 

(1-2) At the i-th iteration either Xi = Xj_i U {xj} or Xj = Xj_i \ {y,z,w} where the 4-tuple 
{xj,y, z,w} violates the four-point condition under metric p. 

Definition 3.2 (Leaf augmentation). Given X' C X, let T be a tree with V(T) n X = XL Given 
a E X \ X' and u, v E X', the a-leaf augmentation of T at {u, u} is the tree T' obtained as 
follows. Let P be the path in T between u and v (which may contain a single vertex if u = v). 
Set r = ( p(a,u ) + p(a,v) — p(u,v))/ 2. Let s be a vertex in P with dp(s,u) = p(a,u ) — r; if no 
such vertex exists then we introduce a new such vertex s by subdividing the appropriate edge in P 
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and update the edge lengths accordingly. In the resulting tree T' we add the vertex a and the edge 
{a,s} if they do no already exist, and we set the length of {a,s} to be r. We call s the stem of a 
(w.r.t. the leaf augmentation). When u = v we say that T' is the a-leaf augmentation of T at u, in 
which case T' is obtained from T simply by adding a as a leaf attached to u, and u is the stem of a. 

In what follows, we set x* to be the nearest neighbor of Xi in Xj_i, that is 

x* = argminp(x, xf), 

X^Xi — \ 

where we break ties arbitrarily. Intuitively, if we can build a new tree T' from Tj_i so that 
{X^ i U {xi},p) can be isometrically embedded in T', then T' is a x^-leaf augmentation of Tj-\ at 
some pair {x*,u}. Our approach will first compute an auxiliary structure, called Xj-orientation on 
Tj_i, to help us identify a potential leaf augmentation. We next check for the validity of this leaf 
augmentation. The key is to produce this candidate leaf augmentation such that if it is not valid, 
then we will be able to find a 4-tuple violating the four-point condition from it quickly. 

Definition 3.3 ((a, u, x)-orientation). Let X' C X and let T be a tree with V[T ) fll = X'. Let 
a G X \ X' and u, v G X'. The (a, u, u)-orientation of T is a partially oriented tree T obtained as 
follows: Let T' be the a-leaf augmentation of T at {u, v}, and let a' be the stem of a. We orient 
every edge in P uv n E{T') towards o', where P uv is the unique path in T between u and v. All other 
edges in E{T) \ P uv remain unoriented. 

If E(T) / E(T ') then there exists a unique edge in e G P uv \ E{T') (which is subdivided in T'); 

this edge e remains undirected in T. We call this edge the sink edge w.r.t. {u,v}. If there is no 

sink edge, then there is a unique vertex in P uv with no outgoing edges in P uv , which we call the 
sink vertex w.r.t. {u, v}. Note that the sink is the simplex of smallest dimension that contains the 
stem of a w.r.to the leaf augmentation at {u,v}. 

See the right figure for an example: where s is stem of a in the leaf 
augmentation at {u, v}. The thick path P uv is oriented, other than the sink 
edge (the one that contains the stem s). 

Definition 3.4 (xj-orientation). An Xj-orientation of T t -\ is any partial ori¬ 
entation T,;_ j of Ti-i obtained via the following procedure: Consider any or¬ 
dering of Xi- 1 , say {vi,V 2 , ■ ■ ■ ,v#} = Xi- 1 . Start with Tj_i = Tj_i, i.e. all 
edges in T t -\ are initialized as undirected and we will iteratively modify their 
orientation. Process vertices vi,... ,V( in this order. For each Vj, denote by 
Pj the path in Tj _\ between Vj and x*. Traverse Pj starting from Vj until we reach either x* or an 
edge which is already visited. For each unoriented edge we visit, we set its orientation to be the one 
in the (xi,Vj,x*)-orientation ofTi-\. An edge that is visited in the above process is called masked. 

Since the above procedure is performed for all leaves of T,;-i, an Xj-orientation will mask all 
edges. However, a masked edge may not be oriented, in which case this edge must be the sink edge 
w.r.t. {vj,x*} for some Vj G 4Q_i. 

Definition 3.5 (Sinks). Given an Xi-orientation T of tree T, a sink is either an un-oriented edge, 
or a vertex v G V(T) such that all incident edges have an orientation toward v. The former is also 
called a sink edge w.r.t. T and the latter a sink vertex w.r.t. T. 

It can be shown that each sink edge/vertex must be a sink edge/vertex w.r.t. {vj,x*} for some 
Vj G Xi- 1 , and we call Vj a generating vertex for this sink. 
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An Xj-orientation may have multiple sinks. We further augment the Xj- 
orientation to record a generating vertex Vj for every sink (there may be multiple 
choices of vj for a single sink, and we can take an arbitrary one). We also re¬ 
mark that a sink w.r.t. some {vj,x*} may not ultimately be a sink for the global 
Xj-orientation: see the right figure for an example, where s is a sink vertex w.r.t. 
{vj,x*}, but not a sink vertex for the global Xj-orientation. 

The proofs of the following two results can be found in Appendix [A} 



Lemma 3.6. An Xi~orientation ofT{-\ (together with a generating vertex for each 
sink) can be computed in O(i) time. 


Lemma 3.7. 


Any Xi-orientation T)_i of T % _\ has at least one sink. 





Figure 1: (a) An illustration for Case 1 in Lemma [3.8| Solid edges are from Tj_ ;1 and the dotted edge 
connecting Xj to its stem s is in L. (b) Case 2 when y = s. (c) If p(xi,w ) > dx(xi,w), then we can find z 
the processing of which during the construction of Ti_i assigns the orientation of edge ( y,y'). 


Lemma 3.8. For any i E {2, ... ,n}, given 7 )-1 , we can compute Ti and Xi satisfying invariants 
(1-1) and (1-2) in 0(i) time. 


Proof. It suffices to show that in 0(i ) time we can either find a 4-tuple of points in W_i U {xj}, 
that violates the four-point condition, or we can compute a tree having a shortest-path metric that 


agrees with p on X,_i U {xj}. By Lemma 3.6, we can compute an x-orientati on T j-\ of X)_i in 


0(i ) time. Consider any sink of T,_i (whose existence is guaranteed by Lemma 3.7), and let u be 
its associated generating vertex; u must be in X,_i. Let L be the Xj-leaf augmentation of Tj_i at 
{x*,u}, and let di denote the shortest path metric on the tree L. 

Since L is the Xj-leaf augmentation of T, we have for all w, w' E W_i, dz,(w, w') = dr z _ t (w, w') = 
p(w,w') (the last quality is because (Xj_i,p) embeds isometrically into Tj_i). Thus di may only 
disagree with p on pairs of points Xj, v, for some v E W_i. We check in 0{i) time if, for all v E Xj_i, 
we have di,(xi,v) = p(xi,v) via a traveral of L starting from the stem of Xj in L. If the above 
holds, then obviously (A*_i U {xj},p) embeds isometrically into L. We then set Xi = Xj_i U {x,} 
and output T* = L. Otherwise, let w E A'j_i be such that dL,{xi,w) / p(xi,w). We now show that 
we can find a 4-tuple including x* that violates the four-point condition in constant time. 

Let s be the stem of Xj in L. Consider L as rooted at x* and let y E V ( L) be the lowest common 
ancester of u and w. Note that y must be a vertex from V(Tj_i) too. Let P vv ' denote the unique 
path in L between any two v and v'. The vertex y must be in the path P ux *. 
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,se 1: y ^ s. In this case, y is either in the interior of path P su or of path P sx *. Assume 
w.o.l.g. that y is in the interior of P su \ the handling of the other case is completely sym¬ 
metric. See Figure [I] (a) for an illustration. Since di, is a tree metric, we know that the 
4-tuple {xi, x*,u,w} should satisfy the four-point condition under the metric cLl- Using the 
alternative formulation of four-point condition in Definition 2.2 we have that the largest two 
quantities of the following three terms should be equal: 


d L (xi,x*) + d L (u,w), d L (xi,u) + d L (x*,w), d L (xi,w) + d L (x*,u). (5) 

For this specific configuration of y, we further have: 

d L (xi,x*) + d L (u,w) < d L (xi,u ) + d L (x*,w) = d L (xi,w) + d L (x*,u). (6) 


On the other hand, by construction, we know that agrees with p on X,;_ i. Furthermore, 
since L is the x,;-leaf augmentation of Tj_i at {u, x*}, we have that d^Xi^u ) = p(xi,u) and 
d L (xi,x*) = p(xi,x*). Hence 0 can be rewritten as 

p{xi,x*) + p(u,w) < p(xi,u) + p(x*,w) = d L (xi,w) + p(x*,u). (7) 

If p(xi,u>) / di(xi,w ), then the largest two quantities of 

P(xi, x*) + p(u, w), p(xi, u) + p(x*, w), p{xi, w) + p{x*,u) 

can no longer be equal as p(xi,x*) + p(u,w ) < p(xi,u) + p(x*,w). Hence the 4-tuple 
{xi,x*,u,w} violates the four-point condition under the metric p (by using ([3])). 

? 2: y = s, in which case s must be a sink vertex: see Figure [I] (b) for an illustration. For this 
configuration of y, it is necessary that 


p(xi,x*) + p(u,w) = p(xi,u ) + p(x*,w) = d L (xi,w) + p(x*,u). (8) 

Hence if p(xt,w) > di,(xi,w), then the 4-tuple {xi,x*,u,w} violates the four-point condition 
under the metric p because 


p(xi,x*) + p(u,w) = p(xi,u) + p(x*,w) < p{xi,w ) + p(x*,u). 


What remains is to find an violating 4-tuple for the case when p(xi,w) < dL(xi,w). 


Now imagine performing the Xj-leaf augmentation of Tj_i at {w,x*}. We first argue that the 
stem s' of Xi w.r.t. {w,x*} necessarily lies in P wy in T t -\. Let r w = (p(xi,x*) + p(xt,w ) — 
p(x*,w))/2. In the augmented tree L, di{xi,y) = (p(xi,x*) +p(xi,u) — p(x*,u))/2. Combing 
Q and p(xi,w) < di,(xi,w) we have that r w < di,(xi,y). On the other hand, following 
the position of s' is such that dx i _ 1 (x*, s') = p(x*,Xi) — r w , while the position 
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Definition 

of y = s was that dTi_ 1 (x*,s) = p(x*,Xi) — dfjxi, y). It then follows that s' must lie in the 
interior of path P wy . 


Since the stem of Xi w.r.t. {w,x*} is in P wy , it means that before we process w in the 
construction of the Xj-orientation Tj_i, there must exist some other leaf z E V(Tj_i) such 
that the process of z assigns the orientation of the edge (y, y') C P wy to be towards y; See 
Figure [T] (c). This is because if no such z exists, then while processing w, we would have 




oriented the edge (y, y') towards stem s ', thus towards y', as the stem s' is in P wy i. The point 
z can be identified in constant time if during the construction of X)_i, we also remember, for 
each edge, the vertex the processing of which leads to orienting this edge. Such information 
can be easily computed in 0(i) time during the construction of T,;_i. 

Now consider If di,(xi,z ) = p(xi,z), then one can show that y = s is necessarily the stem 
for Xi w.r.t. {xi,z} as well (by simply computing the position of the stem using Definition 

new 4-tuple), which in turn means that this 4-tuple violates the four-point condition. Hence 
we are done. 

If z) p(xi, z), then since we orient the edge (y, y') towards y during the process of leaf 

z, the stem of x,; of the leaf augmentation at {x^, z} is in the path P yx *. By an argument similar 
to the proof that s' is in the interior of P wy above, we can show that p(xi, z) > di,(xj, z). Now 
consider the 4-tuple {xj, x*,z, u }: this leads us to an analogous case when p(xj, w ) > di(xj, w) 
for the 4-tuple {xi,x*,u,w}. Hence by a similar argument as at the beginning of Case 2, we 
can show that {xj, x*, z, u} violates the four-point condition under metric p. 

Putting everything together, in either case, we can identify a 4-tuple U, which could be 
{xj, x*, u, w}, {xi,x*,z,w}, or {xi,x*,z,u} as shown above, that violates the four-point condi¬ 
tion under metric p. We simply remove these four points, adjust the resulting tree to obtain T % and 
set Xi = Xi-i \ U. The overall algorithm takes 0(i) time as claimed. This proves the lemma. □ 

Theorem 3.9. There exists a 4 -approximation algorithm for minimum outlier embedding into trees, 
with running time 0(n 2 ). 

Proof. By Lemma |3.8| and induction on i = 1,..., n, it follows immediately that we can compute 
T n in time 0(n 2 ). By invariant (1-1), the output (X n , p) is a tree metric as it can be isometrically 
embedded into T n . Furthermore, by invariant (1-2), each 4-tuple of points we removed forms a 
violation of the four-point condition, and thus must contain at least one point from any optimal 
outlier set. As such, the total number of points we removed can be at most four times the size of 
the optimal solution. Hence our algorithm is a 4-approximation as claimed. p| 

3.3 Approximating outlier embeddings into 

In this section, we present a 2-approximation algorithm for the minimum outlier embedding problem 
into the Euclidean space in polynomial time, which matches our hardness result in Appendix [b] 
Given two points p, q £ M d , let d^(p, q) = \\p — < 7 11 2 denote the Euclidean distance between p and q. 

Definition 3.10 (d-embedding). Given a discrete metric space X = (X,p), an d-embedding of 
X is simply an isometric embedding ip : X —>• of X into (M d ,d£j); that is, for any x,y E X, 

p(x,y) = du(</?(x), p(yf). We say that X is strongly d-embeddable if it has a d-embedding, but 
cannot be isometrically embedded in In this case, d is called the embedding dimension of X. 

The following is a classic result in distance geometry of Euclidean spaces, see e.g HU37]. 

Theorem 3.11. The metric space X = (X, p) is strongly d-embeddable in if and only if there 
exist d + 1 points, say Xj = {xo,..., x^}, such that: 

(i) (X(i, p) is strongly d-embeddable; and 

(ii) for any x, x' 6 X \ X c j, [X ( ] U {x, x'}, p) is d-embeddable. 


3.2). In this case, considering the 4-tuple {xj, x*, z, w}, we are back to Case 1 (but for this 
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Furthermore, given an m-point metric space (X,p), it is known that one can decide whether 
(. X , p) is embeddable in some Euclidean space by checking whether a certain mxm matrix derived 
from the distance matrix p is positive semi-definite, and the rank of this matrix gives the embedding 
dimension of X; see e.g. [36]. 

Following Theorem 3.11, one can easily come up with a (d + 3)-approximation algorithm for 
minimum outlier embedding into M rf , by simply checking whether each (d + 3)-tuple of points is d- 
embeddable, and if not, removing all these d +3 points. Our main result below is an 2-approximation 
algorithm within the same running time. In particular, Algorithm [l] satisfies the requirements of 
Theorem |3. 12 and the proof is in Appendix [A} 


Theorem 3.12. Given an n-point metric space (X,p), for any d> 1, there exists 2-approximation 
algorithm for minimum outlier embedding into M rf , with running time 0(n d+s ). 


Algorithm 1 2-approximation outlier embedding in M rf . 

Input: An n-point metric space (X,p) 

Output: A set of outliers K 

Initialize the set of candidate outlier sets C to be empty. 

(Step-0) For each d+ 1 number of distinct points Y ( i = {y 0 , • • •, Vd} C X, perform the following: 

Initialize sets Z and K to be the empty set. 

(Step-1) Check whether (Yd, p) is d-embeddable in M d : If not, return to (Step-0). Otherwise, 
compute its embedding dimension d!\ note, d! < d. 

(Step-2) For each remaining point x G X \ Yd, check whether (Yd U {s}, p) is d'-embeddable: 
If yes, insert x to Z; otherwise, insert x to the outlier set I\. 

(Step-3) Construct a graph G = ( Z,E ), where (z,z') 6 E if (Yd U {z,z'},p) is not d'- 
embeddable. Compute a 2-approximation Z' C Z of the vertex cover of G. Set I\ = KUZ', 
and we add the set K to the collection of candiate outlier sets C. 

(Step-4) Let I\ be the set from C with smallest cardinality. We return K as the outlier set. 


Hardness results. In Appendix [Bj we show that the minimum outlier embedding problems into 
ultrametrics, trees and Euclidean space are all NP-hard, by reducing the Vertex Cover problem to 
them in each case. In fact, assuming the unique game conjecture, it is NP-hard to approximate 
each of them within 2 — v, for any positive v. For the case of minimum outlier embedding into 
Euclidean space, we note that our 2-approximation algorithm above matches the hardness result. 

4 Bi-criteria approximation algorithms 

4.1 Bi-criteria approximation for embedding into ultrametrics 

Let T = (A, E ) be a tree with non-negative edge weights. The ultrametric induced by T is the 
ultrametric U = (X,S) where for every x,y £ X we have that 5(x,y) is equal to the maximum 
weight of the edges in the unique x-y path in T (it is easy to verify that the metric constructed as 
such is indeed an ultrametric [20]). Given an metric space X = (X , p) , we can view it as a weighted 
graph and talk about its minimum spanning tree (MST). The following result is from [20] . 
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Lemma 4.1 (Farach, Kannan and Warnow [20]). Let X = (X,p) be a metric space and let U* be 
an ultrametric minimizing \\X — hi* ||oo- LetU be ultrametric induced by a MST of X. Then there 
exists a > 0, such that U* =U + a. 

In particular, let f3 = \\U — A’|| 00 . Then U* = U + f3/2. This further implies that \\U* — X\\oo = 
\\U-X\\ 00 /2 = P/2. 

Theorem 4.2. There exists a polynomial-time algorithm which given an n-point metric space 
X = (X, p), e > 0, and k > 0, such that X admits a ( e,k)-outlier embedding into an ultrametric, 
outputs a {0{e\og(n)),2>k)-outlier embedding into an ultrametric. 

Proof. For simplicity, assume that the diameter A(T) = 1. The algorithm is as follows. We first 
enumerate all triples {x, y, z} £ (^). For any such triple, if p(x,y ) > ma x{p(x, z), p(z, y)} + 2s, 
then we remove x, y , and z from X. Let X' be the resulting point set. We output the ultrametric 
W induced by an MST of the metric space (X', p). This completes the description of the algorithm. 

It suffices to prove that the output is indeed an (0(s log(n)), 3fc)-outlier embedding. Let K* C 
X, with | A'*| < k, be such that ( X \ K*,p ) admits a (e, 0)-outlier embedding into an ultr ame tric. 
Let {x,y,z} £ (^) be such that p(x,y) > max{p(x, z), p(z, y)} + 2e. It follows by Lemma 4.1 that 
({x, y, z}, p) does not admit a (e, 0)-outlier embedding into an ultrametric. Thus, A"*n{x, y , z\ / 0. 
It follows that \X \ X'\ < 3 • | K*\ < 3 k. In other words, the algorithm removes at most 3 k points. 

It remains to bound the distortion between U’ and (X',p). Let T be the MST of (X', p) 
such that the algorithm outputs the ultrametric hi' = (X',5) induced by T. We will prove by 
induction on i > 0, that for all x,y £ X' , if the x-y path in T contains at most 2* edges, then 
p(x,y) < 5{x,y) + 2i ■ e. 

For the base case i = 0 we have that {x, y} £ E{T). Since T is the minimum spanning tree of 
(X',p), it follows that S(x,y) = p(x,y), proving the base case. For the inductive step, let x,y £ X’ 
such that the x-y contains at most 2* edges, for some i > 1. Let w G X' be such that w is in 
the x-y path in T, and moreover the x-w and w-z paths in T have at most 2 l ~ 1 edges each. Since 
{x, y,w} £ X', it follows that the triple {x, y,w} was not removed by the algorithm, and thus 

p{x, y) < max{p(x, w),p{w, y)} + 2e. (9) 

By the inductive hypothesis we have 

p{x, w) < 5(x, w) + 2{i — 1 ) ■ e, and p{w, y) < 5(w, y) + 2{i — 1 ) • e. ( 10 ) 

By Q and (10) we get 

p(x, y) < max{<5(x, w ), 5(w, y)} + 2{i — 1) ■ e + 2 e < <5(x, y) + 2i ■ e. 

Hence hi’ is an (2elogn, 0)-embedding of ( X', p ). This, together with the bound on \X \ X'\ 
concludes the proof. □ 


4.2 Bi-criteria approximation for embedding into 

Theorem 4.3. Given an n-point metric space X = (X,p), if X admits a (e, k*)-outlier embedding 
in M''*, then we have an algorithm that outputs an (O(yfs), 2k*)-outlier embedding of X in M. d in 
0(n d+3 ) time. Here the big-O notation hides constants depending on the dimension d. 

We only state the main result here, and the details can be found in Appendix [C| In particular, 
the high level structure of our algorithm parallels that of Algorithm [lj which intuitively is an 
algorithm for the special case when e = 0. However, the technical details here are much more 
involved so as to tackle several issues caused by the near-isometric embedding. 
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A Missing details from Section [3] 

Proof of Lemma |3.6| . Start with an arbitrary ordering of . and follow the procedure laid-out 
in Definition |3.4| Specifically, for each Vj, we can compute the distance r 3 = ( p(vj,Xi ) + p(x*, Vj) — 
p(xi,x*))/ 2 in constant time. Hence we know the distance lj = p(vj,Xi ) — r 3 from Vj to the stem 
of Xi w.r.t. {vj,x*}. Now, starting from Vj, we follow the unique path towards the root x*, till 
we meet an masked edge or we meet the root x*. (In other words, we do not traverse the entire 
path Pj from Vj to x*.) For any edge we traverse, first we orient it towards x* till we meet an edge 
(w,w') such that p(w,Vj ) < lj and p(w',Vj ) > lj, or a vertex w such that p(w,Vj ) = lj. In this 
case, we set this edge or vertex to be a sink (as it would contain the stem of Xi at {vj,x*}) and 
associate Vj as its generating vertex. After the sink, we orient each edge to be towards the sink, till 
we meet an masked edge. The entire procedure to process all Vj's takes 0(|H(Tj_i)|) time - since 
all leaves of T)_i are from i, we have that 0(|H(I)_i)|) = 0(|Xj_i|) = 0(i). 

Note that in this procedure, some sink vertex w.r.t. certain {vj,x*} may ultimately not be a 
sink vertex for the final Xj-orientation; recall the figure above this lemma. So we may associate 
generating vertex information for some non-sink vertices. 


Proof of Lemma 3.7, Consider the ordering of leaf vertices of Tj_i, denoted by {ui,... ,vg}, 
that gives rise to the Xj-orientation T)_i. Following the procedure laid out in the proof of Lemma 

vxx~ (from v\ to x* 
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after processing v±, we obtained the (xj, v±, x|)-orientation along the path P v 
in Tj_i). Hence at this point, there must be a sink w.r.t. {m,x*}. If this is a sink edge, since this 
edge is masked, it remain unoriented throughout the entire procedure; that is, this edge will still 
be a sink edge for the final Xj-orientation Tj_j . Hence T has at least one sink, and we are done. 

Otherwise, consider the sink vertex w\ w.r.t. {t>i,x*}: Since w\ is not a sink vertex for Tj_i, 
there must exist an outgoing edge (w±, u) oriented towards u. In other words, the orientation is 
away from the root x*. Follow any outgoing edge from u and continue in this manner: we have to 
either meet a sink edge / vertex (at which point we cannot find another outgoing edge to continue), 
or we reach a leaf Vj of T)-i • All edges we visited during this process are oriented away from the 
root x*. Since when we start with each Vj, we always first orient edges towards the root x*, this 
means such a path cannot lead us to a leaf Vj , and thus we have to stop at a sink of Tj_i. Putting 
everything together, Tj_i has at least one sink. 


Proof of Theorem 


3.12 


We claim that the output outlier set I\ of Algorithm [l] stated in the 
main text 2-approximates the minimum outlier d-embedding of (X, p). Specifically, let K* be an 
optimal outlier set. We will show: (a) (X \K,p ) is d-embeddable; and (b) \K*\ < \K\ < 2|A'*|. 

To prove (a), let Y ( i be the d +1 points in (Step-0) that gives rise to K, and d{< d ) the embedding 
dimension of (Y c j, p). Set Zj = X \ ( K U Yd). (Step-3) ensures that for any two points z,z' G Zd, 
(Yd U {z, z'},p) must be d-embeddable. This is because any pair z,z' that violates this condition 
will form an edge in the graph G constructed in Step-3, and subsequently, the vertex cover will 


contain at least one point from such a pair. Statement (a) then follows from Theorem 3.11 

We now prove statement (b). The left inequality is evident. Let Z* = X \ K*. Since (Z* , p) 
is d-embeddable, by Theorem 3.11[ there exists d* + 1 points, say Y* = {yo,... ,yd*} with d* < d 
being the embedding dimension of Z*, such that (Y*, p) is strongly d*-embeddable. 

Now consider the time when the algorithm enumerates some Yd such that Yd C Z* and Y* C Yd 
in (Step-0); Yd = Y* if d* = d. In the subsequent (Step-2), any point inserted into K violates 
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condition (ii) of Theorem 3.11 and thus must belong to the optimal outlier set K* as well. In 
(Step-3), again by Theorem 3.11 (ii), for any edge (z,z') G E, at least one of its endpoint is 
necessarily in K*. Hence a smallest set of outliers consistent with Y r j would consist of the points 
removed in (Step-2) and the vertex cover of the graph G constructed in (Step-3). Since the algorithm 
computes a 2- approximation of the vertex cover, it follows that the set K at the end of (Step-3) 
contains at most 2|Lf*| points. Since \K\ < \K\, we then have that \K\ < 2\K*\. 

Hence the output outlier set K is a 2-approximation for the minimum outlier embedding of 
( X , p) to as claimed. 

Finally, we analyze the time complexity of our algorithm. Steps 1-3 will be executed 0(n d+l ) 
number of times. For each Y^, (Step-1) takes time polynomial in d due to the discussion below 
Theorem 3.11 (Step-2) performs 0{n) number of embedding test, each for 0(d) points. Hence it 
takes 0(nd u l 1 !) total time. In (Step-3), it takes 0(n 2 d°^) time to construct the graph G = (Z, E). 
A 2-approximation of the vertex cover can easily be computed in 0(|V| + \E\) time |16j . Putting 
everything together, the running time is 0(n d+3 d°^) = 0(n d+3 ), where the big-0 notation hides 
terms polynomial in d. 


B Hardness results 

In this section, we show that the minimum outlier embedding problems into ultrametrics, trees, 
and Euclidean space are all NP-hard, by reducing the vertex cover problem to them in each case. 
In fact, it is NP-hard to approximate each of them within 2 — e, for any e > 0, unless the unique 
game conjecture is true. For the case of minimum outlier embedding into Euclidean space, we note 
that our 2-approximation algorithm from the previous section almost matches the hardness result 
here. 

All the hardness results are obtained via reduction from vertex cover. The high-level approach 
is very similar, although the specific reduction is different in each case. Below we present the result 
for tree metric first. 

Theorem B.l. The problem of minimum outlier embedding into a tree metric is NP-hard. Fur¬ 
thermore, assuming the Unique Games Conjecture, it is NP-hard to approximate within a factor of 
2 — e, for any e > 0. 

Proof. We use a reduction from Vertex Cover. This problem is known to be NP-hard. Furthermore, 
assuming the Unique Games Conjecture, it is also hard to approximate within a multiplicative factor 
of 2 — e for any e > 0 [29]. 

Let G = ( V,E ) be an instance to Vertex Cover. The goal is to find the smallest subset of 
vertices of V covering all edges in E. We construct a discrete metric space (X, dx) as follows: Let 
v be an arbitrarily small positive number. 

• For any Vi G V, add two nodes x t , yi G X. Set X = {xi, 2/*}ie[i,n] U {o}. 

• Set dx(xi,yi ) = 1, dx(o,xf) = 2, and dx(o,yi ) = 1 for each i G [1 ,n]. Set dx(yi,Vj) = 2 and 
dx(xi,yj) = 3 for i / j. 

• Set dx(xi,Xj ) = 4 — v if (vi,Vj) G E is an edge in E\ otherwise, set dx(xt,Xj) = 4 if 
(vi,vj) E. 

It is easy to verify that the above description indeed specifies a metric space. Intuitively, (X,dx) 
is close to the tree metric induced by a star-shaped tree: o is the root, tree edges are (o, xf) and 
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(xi, yi), for i E [1, n], and each edge has weight 1. Specifically, we claim that the optimal solution to 
minimum outlier embedding of (X,dx) into a tree-metric has the same size as the optimal vertex 
cover for G = (V,E). 

Indeed, let K* C X be the smallest set of outliers that one needs to remove so that the metric 
space restricted to X \ K* is a tree metric. Let C* C V denote the optimal vertex cover for 
G = (V,E). It is easy to see that \K*\ < |(7*|, as removing the nodes x% corresponding to vertices 
in C* will result in a tree metric induced by the star rooted at o described above. We next show 
that |<7*| < | K*\. 

Given K*, construct a subset V C V as V = {vi E V \ Xi E K* or y i E K*}. By construction, 
\V\ < \K*\. We now show that V forms a vertex cover for G = (V,E), which in turn implies 
|<7*| < \K*\. Indeed, suppose there exists an edge ( Vi,Vj ) E E such that Vi V and vj ^ V. 
This means that Xi,yi,Xj,yj El \ K*. However, it is easy to check that these four points violate 
the four-point condition as in ([ 2 ]), as dx(xi,yj ) + dx(xj,yi ) = 6, which is strictly larger than 
dx(xi,Xj ) + dx(yj,Vi) = 6 — v, as well as larger than dx{xi,yi ) + dx(yj,Xj) = 4. Contradiction. 
Hence either Vi or Vj must be included in V. Hence 1(7*1 < \V\ < \K*\. 

Combining \K* < |C*| and |(7*| < \K*\, we have \C*\ = | K* \. The claimed hardness results, 
both for computing and for approximating the tree-metric outlier-embedding problem thus follow 
from the hardness of Vertex Cover. □ 


Theorem B.2. The problem of minimum outlier embedding into an ultrametric is NP-hard. Fur¬ 
thermore, assuming the Unique Games Conjecture, it is NP-hard to approximate within a factor of 
2 — e, for all e > 0. 


Proof. As for the case of embedding into trees, we give a reduction from Vertex Cover. Specifically, 
let G = ( V ., E) be an instance of Vertex Cover. We construct a discrete metric space (U, djj) as 
follows: Let v be a sufficiently small positive number. 


• For each m E V, i E [1, n], add a pair of nodes Ui, Wi E U. Hence U = {ui, [l,n]- 

• Set djj(ui,Wi ) = 2u, and djj(ui,Wj ) = 1 , djj(u>i,Wj ) = 1 for any i / j. For i ^ j, set 
djj{ui,Uj ) = 1 — v if {yi, Vj) E E; otherwise, set du(v,i,Uj) = 1 . 


It is easy to verify that the above description indeed specifies a metric. Let K* denote the optimal 
set of outliers so that djj restricted to U\K* gives rise to an ultrametric. Let C* denote the optimal 
vertex cover. Similar to the proof of Theorem B.l we will show that |(7*| = |A'*|. First, it is easy 
to see that by removing those vertices Ui such that Vi E (7*, the resulting metric is an ultrametric. 
Hence \K*\ < |(7*|. It remains to show that |(7*| < \K*\. 

To this end, construct V = {vi E V \ Ui E K* or Wi E K*}; obviously, |V| < \K*\. We claim 
that V forms a vertex cover for G = (V,E). Specifically, assume this is not the case and there 
exists an edge (uj, vj) E E such that Vi fiV and Vj ^ V. Then m , Wi , Uj E U \ K*. However, these 
three points violate the three-point condition for ultrametrics as in 0 . as djj(wi,Uj ) = 1 , which is 
strictly larger than rna x{du(vJi,Ui) = 2 v,d(j(uj,Ui) = 1 — i'}, which is a contradiction. Hence V is 
a vertex cover and thus |(7*| < |V| < \K*\. Overall, |(7*| = \K*\ and the theorem then follows. □ 


We next present a hardness result for outlier embedding into constant-dimensional Euclidean 
space. 
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Theorem B.3. For any d > 2, the problem of minimum outlier embedding into d-dimensional 
Euclidean space is NP-hard. Furthermore, assuming the Unique Games Conjecture, it is NP-hard 
to approximate within a factor of 2 — e, for any e > 0. 

Proof. We first present the construction for d = 2, and then discuss how to extend it to the case 
d > 2. As before, we give a reduction from Vertex Cover. Specifically, given a graph G = (V,E), 
we construct a discrete metric space (X,dx) as follows: Let v be a sufficiently small positive real 
number. Set X = {xi ,..., x n , yi,..., y n , Zi,..., z n }. 

• We define: 

1. for any i £ [l,n], Xi = (i,-l),yi = (i, l),and z* = (i, 0). 

• We assign distances dx as follows: 

1. d x (xi,Zi) = 1, d x (yi,Zi) = 1 and d x (xi,yi ) = 2, for any i £ [l,n]. 

2. For any i,j £ [1, n], for a, b £ {x,y}, dx{ai,bj ) = ||aj — bj ||. 

3. For any i,j £ [l,n], d x {zi,Xj) = ||Zi - xj\\, d x (zi,yj ) = ||Zi - yj\\. 

4. For i / j, if (vi, Vj) £ E, then dx{zi, Zj) = |j — i\ — u\ otherwise, set dx(zi, Zj) = | j — i |. 

In other words, other than Z{ and Zj potentially deviates from the distance given by their coor¬ 

dinates, the distance between all other pairs of points are consistent with their coordinates. It is 
easy to verify that (A, dx) as described above satisfies triangle inequality and is a valid metric. 
Let K* denote the optimal set of outliers so that dx restricted to X \ K* can be embedded in 
R 2 isometrically. Let C* denote the optimal vertex cover for graph G = (V,E). We now show 
| if* | = |C*|. 

Specifically, first note that given C *, we can construct the set K containing all y t such that 
Vi £ C *. It is easy to see that the coordinates for the remaining points X \ K are consistent with 
their pairwise distances. Hence \K*\ < \K\ = |(7*|. We next show that |C*| < |if*|. 

Consider K*. We construct the set V = {vi \ Xi £ I\* or yi £ K* or z % £ K*}. We now 
argue that V is a vertex cover for the graph G = (V,E). Assume that this is not true, and that 
there exists an edge ( Vi,Vj ) £ E such that Vi ^ V and Vj ^ V. This means that the 6 points in 
{xi,yt, Zi, Xj, yj, Zj} must all belong to X \ K*. However, we argue that their pairwise distances are 
not consistent. In particular, consider the 3 points Xi,yi,Xj. They are isometrically embeddable 
into M 2 (but not into R 1 ). Hence once their coordinates are fixed, there is only a unique position 
possible for any other point p given its distance to these three points. In other words, we can 
uniquely embed z. t , as well as Zj into R 2 w.r.t. the current coordinates of {xi,yi,Xj} (which form 
a valid isometry consistent with dx). However, the Euclidean distance between z t and Zj is not 
consistent with dx- Hence the points {x t , y,:, xy, Zi, z 3 } are not isometrically embeddable into R 2 , 
which contradicts our assumption that all of them are in X \ I\*. It then follows that V has to be 
a vertex cover, and 1(7*1 < \V\ < \K*\. This completed the proof for d = 2. 

We next discuss how the above construction can be generalized to the case d > 2. In the 
construction of (X,dx) we map each £ V to a sequence of d + 1 points in X 

contained in the subspace R ci_1 x {i}, where the first d points form a canonical (d — l)-simplex, 
while the last point xf^ is the centroid of this simplex. The rest of the argument remains essentially 
identical so we omit the details. □ 
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C Bi-criteria approximation for embedding into W l 

As before, let dg denote the Euclidean distance in R rf . Given a point x £ R rf and a set A C R rf , 
the distance from x to A is defined as d(x, A) = Ae{x, y ). 

Definition C.l (Near-isometric d-embedding). Given a metric space X = (X, px), we say that 
a map ip : X —» R rf is a (5-near-isometric embedding of X into R rf if for any x,x' £ X, we have 
that | p(x,x') — cIe(p(x), p(x'))\ < 5. In this case, we also say that X admits a 5-near-isometric 
d-embedding, and ( p(X),dE ) is a 5-approximation of X. 

We note that the parameter 5-above is not normalized w.r.t. the diameter of the set; this is for 
the convenience of the latter argument in this section. Before we describe our algorithm, we need 
some more definitions (following the notations of [33). 

Definition C.2 (Normalized sequence). Given an ordered sequence of points U = {uo, u\,..., Ud} C 
R rf , we say that U is normalized if uq = o = (0,... ,0), and if Ui £ R+ for all 1 < i < d. 

Given U C A C R rf , if f : A —>• R rf is a map such that f(U) is normalized, then we say that the 
map f is normalized at U. 

Definition C.3 (Maximal sequence). Let k < d. Given an ordered sequence of points U = 
{uq, ..., Uk} from a set A C R d , we say that U is maximal in A if: (i) de(uo,ui) = A(A), 
and (ii) for any 2 < i < k, the distance d(«j,aff{uo, • • •, Ui-i}) is maximal in A. Here, aff B is the 
affine subspace ofW* spanned by points in set B C R d . 


Description of the algorithm. Assume that the input n-point metric space X = ( X , p) admits 
a (e, fc*)-outlier embedding into R d . We now present an algorithm, described in Algorithm [ 2 } to 
find an approximation of such an embedding. The high level structure of this algorithm parallels 
that of Algorithm [lj which intuitively is an algorithm for the special case when e = 0. However, the 
technical details here are more involved so as to tackle several issues caused by the near-isometric 
embedding. In this algorithm, Cd is a positive constant depending only on the dimension d. 

For the time being, for the sake of simplicity of presenting the main ideas, let us assume that we 
can implement (Step-1,2) (i.e, the computation of Pd and p x )) in polynomial time. We will address 
how to modify Algorithm [2] to achieve this later in this section. Our main result is the following: 

Theorem C.4. Given an n-point metric space X = (X, p), if X admits a (e, k*)-outlier embedding 
in M d , then Algorithm^ returns an outlier set K that witnesses an (0(^/e), \K\)-outlier embedding 
of X in M. d with \K\ < 2k*. Here O(y^e) hides constants depending on the dimension d. 


Proof of Theorem 


C.4 


Let Z = X \ K. By (Step-2) and (Step-3), for any two points z,z' £ Z, 
we computed p z and p z > £ in (Step-2) such that | p(z, z') — dE{p z ,Pz')\ < Cdy/eX(} *f) + eA(X). 
Hence ( Z,p) indeed has an 0(y / £A(A))-near-isonretric embedding in R rf . 

It remains to show that \K\ < 2k*. Let K* be an optimal set of outliers such that \K*\ = k* and 
(. X \ K* , p) admits a £A(X)-near-isometric d-embedding, say ip : X \ K* R rf . Set Z* = X \ K*. 
For simplicity, we assume that there does not exist d! < d such that (X \ K* , p) admits a eA(A)- 
near-isometric cf-embedding; the case where such d! < d exists can be handled similarly. 

Let Y* = {yo, • ■ ■ ,yd} be the ordered sequence such that <p(Y*) is maximal in ip(Z*). We 
assume w.l.o.g. that p(Z*) is also normalized in R rf ; if not, there exists an isometry T : R rf —> R rf 
so that T(ip(Y*)) is normalized, and we can simply take p as T o ip. 
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Algorithm 2 Bi-criteria approximation for embedding into W i . 

Input: An n-point metric space (X,p), a parameter 0 < e < 1, dimension d 
Output: A set of outliers K C X 

Initialize the set of candidate outlier sets C to be empty. 

(Step-0) For each subset of d + 1 distinct points = {yo,..., yd} C X, perform the following: 
Initialize sets W and K to be the empty set. 

(Step-1) Compute an normalized sequence Pd = {po,Pi, ■ ■ ■ ,Pd} C such that ( Pd-.dE ) 
is an (eA(A))-approximation of ( Yd, p ). If such a Pd does not exist, return to (Step-0). 
Otherwise, compute the smallest dimension d! <d for which such a Pd exists. 

(Step-2) For each remaining point x € X \ Yd, compute p x G W 1 ' satisfying that \p(x,yi) — 
d e [px iPi)\ 5: Cdy/e A (A) + eA(X) for any i G [0, d\. If such a p x can be found, insert the 
pair (x,p x ) into the set W; otherwise, insert x to the outlier set K. 

(Step-3) Construct the graph G = (W, E) where, for any two w = ( z,p),w' = ( z',p') G W, 
(w,w') G E if and only if | p(z,z') — dE(p,p')\ > Cd^/sA(X) + eA(X). Compute a 2- 
approximation W’ C W of the vertex cover of G. Set K = K U {z | ( z,p ) G W’}, and add 
the set K to the collection of candiate outlier sets C. 

(Step-4) Let K be the set from C with smallest cardinality. We return K as the outlier set. 


Consider the time when our algorithm enumerates Yd = Y* in (Step-0). In the subsequent (Step- 
1), the algorithm will be able to find an normalized sequence Pd as claimed, since p(Yd) satisfies 
the requirements. Following our earlier assumption on Z*, d' = d for Pd■ In (Step-2), for a point 
x £ X \ Z*, we claim that there exists p x G such that (PdU {p x }, dg) is an 2e Ax-approximation 
of (Yd U {x},p). To prove this, we need to use the following result proved in [3]. 


Proposition C.5 (Section 2.6, [3]). For any fixed d > 1, let A = {0,14 ,... ,uj,x} C M d be such 
that the d-sequence U = (o,u\,... ,ufi) is normalized and maximal in A. Let f : A —> R. d be a map 
such that ( f(A),dE ) is an 5 A(. A)-approximation of (A^e), and f is normalized at U. Then there 
exists a constant c^ depending only on the dimension d such that dE(x, f(x)) < c^VSA(A). 


Consider the image Q* = p(Z*) of an optimal embedding ip : Z* = X \ I\* —»• M. d . Denote by 
Qd, = Uo = tp(yo),qi = p(yi), • ■ •, (Id = v(yd)} and g : Q d P d C R d the map that sends q, to 
Pi, for each i G [0 ,d\. We also use h : Y* —> Pd to denote the eA(A)-near-isometric embedding 
computed in (Step-1). 

For any x G Z*, by the same argument as for the existence of Pd, we can show that there must 
exist a eA(A)-near-isometric embedding h x : Yd U {x} —> M. d of (Yd U {x}, p) such that the ordered 
sequence {h x (yo), ■ ■ ■ ,h x (yd)} is normalized. Furthermore, Proposition C.5 implies the following: 


Claim C.6. For any x G Z* and any h x as described above, dE(<p(x),h x (x)) < 2cdy/eA(X). 


Proof. For simplicity of presentation, set yd+i = x, and consider the set A = {go, • • •, Qd, Qd+i = 
ip(x)} C Q*, and the map g x : A —> defined as g x (qi ) = h x (yi) for any i G [0, d+ 1]. Since both <p 
and h x are £A(A)-near-isometric maps, we have by triangle inequality that: for any i,j G [0, d+ 1], 


I d E (qi,qj) - d E (h x (yi),h x (yj)\ < \d E (qi,qj) - p(yi,yj)\ + \p(yi,yj) - d E (h x (yi),h x (yj))\ < 2 eA(X). 


20 






In other words, this means that the map g x is a 2sA(X)-nearisometry for ( A,d E )• Setting 5 = 
2eA(X) / A(A), it then follows from Proposition |C.5 that 

d E {qd +1 = <p(x), h x (x)) < c d VdA(A) = c d y / 2eA(X)A(A) < 2 c d y/eA(X). 

The last inequality follows that since ip is £ A (X l-nearisometry, we have that A(A) < A (Q*) < 
(1 + e)A(Z*) < 2A(X) for e < 1. □ 

On the other hand, note that the above statement is generic for any x G Z* and any eA(X)- 
near-isometric map h x such that the ordered sequence {h x (yo ),..., h x (y d )\ is normalized. Hence 
we can choose x = y z for any i E [0, d] and choose h x = h computed in (Step-1) h : Y d -E P d that 
gives rise to P d . It then follows from Claim [CTti] that: 

for any i G [0 ,d], d E (<p(yi),Pi = h(yi )) < 2c d ^/eA(X). (11) 

Finally for any x G X \ Y d inspected in (Step-2), set p x = h x {x) for any e A (X) -near-isometry 
h x :Y* U {x} —>• (which must exist as we argued earlier). Combining Claim C.6 Eqn © , the 
triangle inequality and the fact that ip is an £ A (X)-nearisometry, we thus have: 


I p(yi,x) - d E (pi,Px) | < \p{yi,x) - d E (ip(yi),<p(x))\ + | d E (tp(yi),ip(x)) - d E {pi,Px)\ 

< eA(X) + d E (ip(yi),pi) + d E (ip(x),p x ) < eA(X) + 4c d y/eA(X). 

Setting C d = 4 c d , we obtain the following. 

Corollary C.7. For any x G Z*, there exists p x satisfying the requirements in (Step-2). 


( 12 ) 


By a similar argument as in Eqn (12), we can also obtain: 


Corollary C.8. For any z , z' G Z*, let p z and p z > be their corresponding points in computed in 
(Step-2). Then \p(z,z') - d E (p z ,p z >)\ < C d ^eA(X) + eA(X). 

Lemma C.9. \K\ < 2\K*\. 

Proof. First, as argued earlier above Corollary [CTj for any (Step-2) will be able to compute 

the required p x G M d . Let K\ denote the set O at the end of (Step-2). It then follows that K\ C K*. 
Furthermore, by Corollary |C.8[ for any edge ((z,p), ( z',p')) G E in the graph constructed in Step-3, 
at least one of z and z! is necessarily in K*. Hence \K* \ I\\\ is at least the size of the vertex cover 
of G. Since we compute a 2-approximation of the vertex cover, it follows that the set K at the end 
of Step-3 satisfies that \K\ < 2|/l*|. This proves the claim as \K\ < \K\. □ 


This completes the proof of Theorem C.4 


Computational issues. The computation of P d and p x in (Step-1) and (Step-2) require solving 
a system of 0(d) quadratic equations and inequalities, and the solution would require infinite pre¬ 
cision. This can be addressed using real algebraic geometry theory for semi-algebraic sets which 
can output the solutions in arbitrary precision in time exponential in O(d). We can also use the fol¬ 
lowing “griding” strategy: After (Step-1), we discretize the d-dimensional cube [—3A(X), 3A(X)] rf 
where the grid length is r = ^jA(X). In (Step-1) and (Step-2), we look for solutions for P d and 
p x where coordinates fall on grid points. This requires us to relax the distance distortion bound 


by another factor of eA(X). However it will not change the final statement in Theorem C.4 The 
time to compute each near-isonretry embedding is (d)°0 ). Overall, following a similar analysis as 
for Algorithm [lj we conclude with Theorem 4.3 
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